{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Deep Learning with PyTorch Lightning\n",
    "\n",
    "In another notebook, we saw how PyTorch can be used to train a model. In this notebook, we will go over [PyTorch Lightening]( https://lightning.ai/docs/pytorch/stable/). \n",
    "\n",
    "\n",
    "PyTorch Lightning is the deep learning framework with “batteries included” for professional AI researchers and machine learning engineers who need maximal flexibility while super-charging performance at scale.\r\n",
    "\r\n",
    "Lightning organizes PyTorch code to remove boilerplate and unlock scalabily.it While doing so, it still offers full flexibility to try any idea without the boilerplate. It helps you focus on the core parts of your model training and provide you with a standard way to organise yo code.\n",
    "\n",
    "L\n",
    "Let us now create a neural network in PyTorch. We will use this network to train a model to take MNIST data as input and produce the class it belongs to.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Running in Colab/Kaggle\n",
    "\n",
    "If you are running this on Colab, please uncomment below cells and run this to install required dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install lightning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torchvision import datasets, transforms\n",
    "from torch.optim.lr_scheduler import StepLR\n",
    "import matplotlib.pyplot as plt\n",
    "import lightning.pytorch as pl\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MNIST \n",
    "\n",
    "MNIST dataset has images 28x28 pixels  = 784 pixels. \n",
    "\n",
    "We will have 10 units at the output layer to signify the digit (0-9) the image belongs to.\n",
    "\n",
    "Let us first load the data and print some images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a transform to normalize the data\n",
    "transform = transforms.Compose([transforms.ToTensor(),\n",
    "                              transforms.Normalize(0.5, 0.5),\n",
    "                             ])\n",
    "# Download and load the training data\n",
    "trainset = datasets.MNIST('MNIST_data/', download=True, train=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=128, shuffle=True)\n",
    "\n",
    "# Download and load the test data\n",
    "testset = datasets.MNIST('MNIST_data/', download=True, train=False, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=128, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let us load a batch of training data and checkout its shape\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = next(dataiter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([128, 1, 28, 28])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "images.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "image size of `(32,1,28,28)` means that we have 32 images, with each image of size (1x28x28) (channels x height x width)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fcae6f9caf0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbJ0lEQVR4nO3df2xV9f3H8dcF2ytoe1kt7e0dpZYfwiJQHZOuQRlIA3SLAeUP/PEHLAwDK2bAnA6D8kOzbizxBwvDxBGYmfyYiUDEjEWKbePW4qggc5sdJZ1A6I9Jwr1QoBD6+f5BvN9dacFzubfv3vJ8JCfh3nvevR+Odzx32ttzfc45JwAAelg/6wUAAG5OBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJi4xXoBX9XZ2amTJ08qIyNDPp/PejkAAI+cczpz5oxCoZD69ev+PKfXBejkyZPKz8+3XgYA4AYdP35cQ4YM6fbxXvctuIyMDOslAAAS4Hr/nictQOvXr9edd96pW2+9VcXFxfroo4++1hzfdgOAvuF6/54nJUDbt2/XsmXLtHLlSn388ccqKirS9OnT1dbWloynAwCkIpcEEyZMcOXl5dHbly9fdqFQyFVUVFx3NhwOO0lsbGxsbCm+hcPha/57n/AzoIsXL6q+vl6lpaXR+/r166fS0lLV1tZetX9HR4cikUjMBgDo+xIeoC+++EKXL19Wbm5uzP25ublqaWm5av+KigoFAoHoxjvgAODmYP4uuOXLlyscDke348ePWy8JANADEv57QNnZ2erfv79aW1tj7m9tbVUwGLxqf7/fL7/fn+hlAAB6uYSfAaWnp2v8+PGqrKyM3tfZ2anKykqVlJQk+ukAACkqKVdCWLZsmebOnavvfOc7mjBhgl599VW1t7frhz/8YTKeDgCQgpISoDlz5ui///2vXnjhBbW0tOiee+7Rnj17rnpjAgDg5uVzzjnrRfyvSCSiQCBgvQwAwA0Kh8PKzMzs9nHzd8EBAG5OBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADARMIDtGrVKvl8vpht9OjRiX4aAECKuyUZX/Tuu+/W3r17//9JbknK0wAAUlhSynDLLbcoGAwm40sDAPqIpPwM6MiRIwqFQho2bJieeOIJHTt2rNt9Ozo6FIlEYjYAQN+X8AAVFxdr8+bN2rNnjzZs2KCmpiY98MADOnPmTJf7V1RUKBAIRLf8/PxELwkA0Av5nHMumU9w+vRpFRQU6OWXX9b8+fOveryjo0MdHR3R25FIhAgBQB8QDoeVmZnZ7eNJf3fAoEGDdNddd6mxsbHLx/1+v/x+f7KXAQDoZZL+e0Bnz57V0aNHlZeXl+ynAgCkkIQH6Omnn1Z1dbX+85//6K9//asefvhh9e/fX4899liinwoAkMIS/i24EydO6LHHHtOpU6c0ePBg3X///aqrq9PgwYMT/VQAgBSW9DcheBWJRBQIBKyXAQC4Qdd7EwLXggMAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATCT9A+kAXO3ee+/1PLNmzRrPM1OmTPE8I0kDBw70PNPW1uZ5ZsWKFZ5nfve733meQe/EGRAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMcDVs4Aa99tprnmcWLlzoeSYtLc3zjHPO80y8Bg8e7Hlm/fr1nmfi+Ttt3LjR8wySjzMgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEFyNFrxcMBj3PvPjii3E91/z58+Oa86qtrc3zzAcffOB5ZuvWrZ5nJOkf//iH55lPPvnE88zAgQM9zxQXF3ue4WKkvRNnQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACS5Gih4Vz4VF9+/f73kmPz/f84wktba2ep6ZM2eO55n6+nrPM+3t7Z5n7r33Xs8zkvTzn//c88yAAQM8z3z00UeeZ5YuXep5Br0TZ0AAABMECABgwnOAampq9NBDDykUCsnn82nnzp0xjzvn9MILLygvL08DBgxQaWmpjhw5kqj1AgD6CM8Bam9vV1FRkdavX9/l42vXrtW6dev0+uuva//+/brttts0ffp0Xbhw4YYXCwDoOzy/CaGsrExlZWVdPuac06uvvqoVK1Zo5syZkqQ333xTubm52rlzpx599NEbWy0AoM9I6M+Ampqa1NLSotLS0uh9gUBAxcXFqq2t7XKmo6NDkUgkZgMA9H0JDVBLS4skKTc3N+b+3Nzc6GNfVVFRoUAgEN3iffssACC1mL8Lbvny5QqHw9Ht+PHj1ksCAPSAhAboy18y/Oov87W2tnb7C4h+v1+ZmZkxGwCg70togAoLCxUMBlVZWRm9LxKJaP/+/SopKUnkUwEAUpznd8GdPXtWjY2N0dtNTU06dOiQsrKyNHToUC1ZskQvvfSSRo4cqcLCQj3//PMKhUKaNWtWItcNAEhxngN04MABTZkyJXp72bJlkqS5c+dq8+bNeuaZZ9Te3q4nn3xSp0+f1v333689e/bo1ltvTdyqAQApz+ecc9aL+F+RSESBQMB6GUiS9957z/NMd793di01NTWeZyRp8uTJcc31Vs8991xccy+99FKCV9K1FStWeJ75xS9+kYSVIBnC4fA1f65v/i44AMDNiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACY8fxwDcCNGjhzpeSaeC7b3tatax2vs2LE99lzx/HfauHFjElaCVMEZEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABggouRokeNGDHC88zu3buTsJLU88Ybb3iemTNnThJW0rXz5897nmltbU3CSpAqOAMCAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAExwMVL0eg8++KDnmU8++SSu59q+fXtcc17Fc5HQsWPHep5xznmekSSfz+d55pVXXonruXDz4gwIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDhc/FerTBJIpGIAoGA9TKQJJ9//rnnmWAw6HkmLS3N80xPamtr8zzzt7/9zfPMD37wA88z0pX/HXo1atQozzOtra2eZ5A6wuGwMjMzu32cMyAAgAkCBAAw4TlANTU1euihhxQKheTz+bRz586Yx+fNmyefzxezzZgxI1HrBQD0EZ4D1N7erqKiIq1fv77bfWbMmKHm5ubotnXr1htaJACg7/H8iahlZWUqKyu75j5+vz+uHxwDAG4eSfkZUFVVlXJycjRq1CgtWrRIp06d6nbfjo4ORSKRmA0A0PclPEAzZszQm2++qcrKSv3qV79SdXW1ysrKdPny5S73r6ioUCAQiG75+fmJXhIAoBfy/C2463n00Uejfx47dqzGjRun4cOHq6qqSlOnTr1q/+XLl2vZsmXR25FIhAgBwE0g6W/DHjZsmLKzs9XY2Njl436/X5mZmTEbAKDvS3qATpw4oVOnTikvLy/ZTwUASCGevwV39uzZmLOZpqYmHTp0SFlZWcrKytLq1as1e/ZsBYNBHT16VM8884xGjBih6dOnJ3ThAIDU5jlABw4c0JQpU6K3v/z5zdy5c7VhwwYdPnxYv//973X69GmFQiFNmzZNL774ovx+f+JWDQBIeZ4DNHnyZF3r+qV//vOfb2hB6NsKCgo8z9xzzz2eZ9asWeN5Jl579+71PLNx40bPM9u2bfM8E++1hg8ePOh5hguLwiuuBQcAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATPhfv5XKTJBKJKBAIWC8DSKqqqirPM5MmTfI8c+TIEc8zkjRq1Ki45oD/FQ6Hr/kp15wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmbrFeAHAzKiws7JHnWblyZY88DxAPzoAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABNcjBS4QW+88Ybnmfz8fM8z27dv9zyzbds2zzNAT+EMCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwcVIgf9x7733ep6ZP3++5xnnnOeZrVu3ep4BejPOgAAAJggQAMCEpwBVVFTovvvuU0ZGhnJycjRr1iw1NDTE7HPhwgWVl5frjjvu0O23367Zs2ertbU1oYsGAKQ+TwGqrq5WeXm56urq9P777+vSpUuaNm2a2tvbo/ssXbpU7777rt5++21VV1fr5MmTeuSRRxK+cABAavP0JoQ9e/bE3N68ebNycnJUX1+vSZMmKRwOa+PGjdqyZYsefPBBSdKmTZv0rW99S3V1dfrud7+buJUDAFLaDf0MKBwOS5KysrIkSfX19bp06ZJKS0uj+4wePVpDhw5VbW1tl1+jo6NDkUgkZgMA9H1xB6izs1NLlizRxIkTNWbMGElSS0uL0tPTNWjQoJh9c3Nz1dLS0uXXqaioUCAQiG75+fnxLgkAkELiDlB5ebk+/fRTbdu27YYWsHz5coXD4eh2/PjxG/p6AIDUENcvoi5evFi7d+9WTU2NhgwZEr0/GAzq4sWLOn36dMxZUGtrq4LBYJdfy+/3y+/3x7MMAEAK83QG5JzT4sWLtWPHDu3bt0+FhYUxj48fP15paWmqrKyM3tfQ0KBjx46ppKQkMSsGAPQJns6AysvLtWXLFu3atUsZGRnRn+sEAgENGDBAgUBA8+fP17Jly5SVlaXMzEw99dRTKikp4R1wAIAYngK0YcMGSdLkyZNj7t+0aZPmzZsnSXrllVfUr18/zZ49Wx0dHZo+fbp++9vfJmSxAIC+w+fiuSpiEkUiEQUCAetlIMU9/vjjcc394Q9/SPBKurZkyRLPM+vWrUv8QoAkCofDyszM7PZxrgUHADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAE3F9IirQ261atSquuXguDr906VLPMxs3bvQ8A/Q1nAEBAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACa4GCl6vddee83zzMiRI+N6rn//+9+eZ9atWxfXcwE3O86AAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATXIwUPepHP/qR55mFCxd6nmlvb/c8I0kLFiyIaw6Ad5wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmuBgpetTDDz/seSYtLc3zzGeffeZ5RpJqamrimgPgHWdAAAATBAgAYMJTgCoqKnTfffcpIyNDOTk5mjVrlhoaGmL2mTx5snw+X8wWz+e5AAD6Nk8Bqq6uVnl5uerq6vT+++/r0qVLmjZt2lUf/rVgwQI1NzdHt7Vr1yZ00QCA1OfpTQh79uyJub1582bl5OSovr5ekyZNit4/cOBABYPBxKwQANAn3dDPgMLhsCQpKysr5v633npL2dnZGjNmjJYvX65z5851+zU6OjoUiURiNgBA3xf327A7Ozu1ZMkSTZw4UWPGjIne//jjj6ugoEChUEiHDx/Ws88+q4aGBr3zzjtdfp2KigqtXr063mUAAFKUzznn4hlctGiR/vSnP+nDDz/UkCFDut1v3759mjp1qhobGzV8+PCrHu/o6FBHR0f0diQSUX5+fjxLQgp47733PM+UlZV5nvn73//ueUaSioqK4poDcLVwOKzMzMxuH4/rDGjx4sXavXu3ampqrhkfSSouLpakbgPk9/vl9/vjWQYAIIV5CpBzTk899ZR27NihqqoqFRYWXnfm0KFDkqS8vLy4FggA6Js8Bai8vFxbtmzRrl27lJGRoZaWFklSIBDQgAEDdPToUW3ZskXf//73dccdd+jw4cNaunSpJk2apHHjxiXlLwAASE2eArRhwwZJV37Z9H9t2rRJ8+bNU3p6uvbu3atXX31V7e3tys/P1+zZs7VixYqELRgA0Dd4/hbcteTn56u6uvqGFgQAuDlwNWz0qM7OTs8zra2tnmemTZvmeQZAz+JipAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACAibg/kjtZIpGIAoGA9TIAADfoeh/JzRkQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAE70uQL3s0nQAgDhd79/zXhegM2fOWC8BAJAA1/v3vNddDbuzs1MnT55URkaGfD5fzGORSET5+fk6fvz4Na+w2tdxHK7gOFzBcbiC43BFbzgOzjmdOXNGoVBI/fp1f55zSw+u6Wvp16+fhgwZcs19MjMzb+oX2Jc4DldwHK7gOFzBcbjC+jh8nY/V6XXfggMA3BwIEADAREoFyO/3a+XKlfL7/dZLMcVxuILjcAXH4QqOwxWpdBx63ZsQAAA3h5Q6AwIA9B0ECABgggABAEwQIACAiZQJ0Pr163XnnXfq1ltvVXFxsT766CPrJfW4VatWyefzxWyjR4+2XlbS1dTU6KGHHlIoFJLP59POnTtjHnfO6YUXXlBeXp4GDBig0tJSHTlyxGaxSXS94zBv3ryrXh8zZsywWWySVFRU6L777lNGRoZycnI0a9YsNTQ0xOxz4cIFlZeX64477tDtt9+u2bNnq7W11WjFyfF1jsPkyZOvej0sXLjQaMVdS4kAbd++XcuWLdPKlSv18ccfq6ioSNOnT1dbW5v10nrc3Xffrebm5uj24YcfWi8p6drb21VUVKT169d3+fjatWu1bt06vf7669q/f79uu+02TZ8+XRcuXOjhlSbX9Y6DJM2YMSPm9bF169YeXGHyVVdXq7y8XHV1dXr//fd16dIlTZs2Te3t7dF9li5dqnfffVdvv/22qqurdfLkST3yyCOGq068r3McJGnBggUxr4e1a9carbgbLgVMmDDBlZeXR29fvnzZhUIhV1FRYbiqnrdy5UpXVFRkvQxTktyOHTuitzs7O10wGHS//vWvo/edPn3a+f1+t3XrVoMV9oyvHgfnnJs7d66bOXOmyXqstLW1OUmuurraOXflv31aWpp7++23o/v861//cpJcbW2t1TKT7qvHwTnnvve977mf/OQndov6Gnr9GdDFixdVX1+v0tLS6H39+vVTaWmpamtrDVdm48iRIwqFQho2bJieeOIJHTt2zHpJppqamtTS0hLz+ggEAiouLr4pXx9VVVXKycnRqFGjtGjRIp06dcp6SUkVDoclSVlZWZKk+vp6Xbp0Keb1MHr0aA0dOrRPvx6+ehy+9NZbbyk7O1tjxozR8uXLde7cOYvldavXXYz0q7744gtdvnxZubm5Mffn5ubqs88+M1qVjeLiYm3evFmjRo1Sc3OzVq9erQceeECffvqpMjIyrJdnoqWlRZK6fH18+djNYsaMGXrkkUdUWFioo0eP6rnnnlNZWZlqa2vVv39/6+UlXGdnp5YsWaKJEydqzJgxkq68HtLT0zVo0KCYffvy66Gr4yBJjz/+uAoKChQKhXT48GE9++yzamho0DvvvGO42li9PkD4f2VlZdE/jxs3TsXFxSooKNAf//hHzZ8/33Bl6A0effTR6J/Hjh2rcePGafjw4aqqqtLUqVMNV5Yc5eXl+vTTT2+Kn4NeS3fH4cknn4z+eezYscrLy9PUqVN19OhRDR8+vKeX2aVe/y247Oxs9e/f/6p3sbS2tioYDBqtqncYNGiQ7rrrLjU2NlovxcyXrwFeH1cbNmyYsrOz++TrY/Hixdq9e7c++OCDmI9vCQaDunjxok6fPh2zf199PXR3HLpSXFwsSb3q9dDrA5Senq7x48ersrIyel9nZ6cqKytVUlJiuDJ7Z8+e1dGjR5WXl2e9FDOFhYUKBoMxr49IJKL9+/ff9K+PEydO6NSpU33q9eGc0+LFi7Vjxw7t27dPhYWFMY+PHz9eaWlpMa+HhoYGHTt2rE+9Hq53HLpy6NAhSepdrwfrd0F8Hdu2bXN+v99t3rzZ/fOf/3RPPvmkGzRokGtpabFeWo/66U9/6qqqqlxTU5P7y1/+4kpLS112drZra2uzXlpSnTlzxh08eNAdPHjQSXIvv/yyO3jwoPv888+dc8798pe/dIMGDXK7du1yhw8fdjNnznSFhYXu/PnzxitPrGsdhzNnzrinn37a1dbWuqamJrd371737W9/240cOdJduHDBeukJs2jRIhcIBFxVVZVrbm6ObufOnYvus3DhQjd06FC3b98+d+DAAVdSUuJKSkoMV5141zsOjY2Nbs2aNe7AgQOuqanJ7dq1yw0bNsxNmjTJeOWxUiJAzjn3m9/8xg0dOtSlp6e7CRMmuLq6Ousl9bg5c+a4vLw8l56e7r75zW+6OXPmuMbGRutlJd0HH3zgJF21zZ071zl35a3Yzz//vMvNzXV+v99NnTrVNTQ02C46Ca51HM6dO+emTZvmBg8e7NLS0lxBQYFbsGBBn/s/aV39/SW5TZs2Rfc5f/68+/GPf+y+8Y1vuIEDB7qHH37YNTc32y06Ca53HI4dO+YmTZrksrKynN/vdyNGjHA/+9nPXDgctl34V/BxDAAAE73+Z0AAgL6JAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDxf86FsQgWU8JdAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let us plot one image\n",
    "plt.imshow(images[10].numpy().squeeze(), cmap='Greys_r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=784, out_features=192, bias=True)\n",
       "  (1): ReLU()\n",
       "  (2): Linear(in_features=192, out_features=128, bias=True)\n",
       "  (3): ReLU()\n",
       "  (4): Linear(in_features=128, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = nn.Sequential(\n",
    "            nn.Linear(28 * 28, 192), \n",
    "            nn.ReLU(), \n",
    "            nn.Linear(192, 128),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(128,10)\n",
    ")\n",
    "\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forward pass and Calculate Cross Entropy Loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to view the probability of classification of digit\n",
    "def view_classification(img, probs):\n",
    "    probs = probs.data.numpy().squeeze()\n",
    "    fig, (ax1, ax2) = plt.subplots(figsize=(6,7), ncols=2)\n",
    "    ax1.imshow(img.numpy().squeeze())\n",
    "    ax1.axis('off')\n",
    "    ax2.barh(np.arange(10), probs)\n",
    "    ax2.set_aspect(0.1)\n",
    "    ax2.set_yticks(np.arange(10))\n",
    "    ax2.set_yticklabels(np.arange(10).astype(int), size='large');\n",
    "    ax2.set_title('Probability')\n",
    "    ax2.set_xlim(0, 1.1)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAERCAYAAACq8dRTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAnOUlEQVR4nO3de1xUdf4/8NcwwOHmjApeQMdLGpiiZRSEX68bX0H8iuWqYJlYmtquun61G7sZoRWUplt5KVvDS5iWN8rVUPpFVuI1rFU0zIQALxkpg4IjMJ/fH/tltpGZAyJwjofX8/E4j935XM68PIlvPucyoxNCCBAREZGiXJQOQERERCzIREREqsCCTEREpAIsyERERCrAgkxERKQCLMhEREQqwIJMRESkAizIREREKsCCTEREpAIsyERECtLpdJg5c2aj7W/NmjXQ6XQ4fPhwnWOHDh2KoUOH2l7n5+dDp9NhzZo1traXXnoJOp2u0fKRcyzIREQO1BS2ms3DwwOBgYGYOXMmLly4oHQ8Rb366qvYvn270jE0hwWZiEjGggULsH79eixbtgwDBgzAypUrER4ejvLycqWj3bLdu3dj9+7dsmNeeOEFVFRU2LWxIDcNV6UDEBGp2YgRI3DfffcBAKZOnQpfX18sWbIE6enpmDBhQq3xV69ehbe3d3PHbBB3d/c6x7i6usLVlaWiOXCFTER0E/7whz8AAM6cOYPJkyfDx8cHp0+fRnR0NFq1aoVHH30UwL8L87x582AymSBJEoKCgrB48WI4+4K9tLQ0BAUFwcPDAyEhIdi7d69df0FBAf70pz8hKCgInp6e8PX1xbhx45Cfn+9wf+Xl5Zg+fTp8fX1hMBgwadIkXLp0yW7MjdeQHbnxGrJOp8PVq1exdu1a2+n8yZMn44svvoBOp8O2bdtq7WPDhg3Q6XTIzs6Wfa+Wjr/2EBHdhNOnTwMAfH19AQBVVVWIjIzEwIEDsXjxYnh5eUEIgZiYGHzxxReYMmUK7rnnHmRkZOCZZ55BcXExli5darfPL7/8Eps2bcLs2bMhSRJWrFiBqKgoHDx4EMHBwQCAQ4cOYd++fYiLi0Pnzp2Rn5+PlStXYujQocjNzYWXl5fdPmfOnInWrVvjpZdewg8//ICVK1eioKAAWVlZt3ST1vr16zF16lSEhoZi2rRpAIAePXrggQcegMlkQlpaGh5++GG7OWlpaejRowfCw8Mb/L4tgiAiolpSU1MFAJGZmSkuXrwoCgsLxcaNG4Wvr6/w9PQURUVFIj4+XgAQzz//vN3c7du3CwDi5ZdftmsfO3as0Ol04scff7S1ARAAxOHDh21tBQUFwsPDQzz88MO2tvLy8loZs7OzBQCxbt26WrlDQkLE9evXbe2vv/66ACDS09NtbUOGDBFDhgyxvT5z5owAIFJTU21tiYmJ4sZS4e3tLeLj42vlSUhIEJIkicuXL9vafvnlF+Hq6ioSExNrjSd7PGVNRCQjIiIC7dq1g8lkQlxcHHx8fLBt2zZ06tTJNuapp56ym7Nz507o9XrMnj3brn3evHkQQmDXrl127eHh4QgJCbG97tKlC0aPHo2MjAxUV1cDADw9PW39lZWVKCkpQc+ePdG6dWt8++23tXJPmzYNbm5udhldXV2xc+fOBhyF+pk0aRIsFgs2b95sa9u0aROqqqowceLEJntfreApayIiGcuXL0dgYCBcXV3RoUMHBAUFwcXlP2sZV1dXdO7c2W5OQUEBAgIC0KpVK7v2u+66y9b/e3feeWet9w0MDER5eTkuXryIjh07oqKiAsnJyUhNTUVxcbHdtejS0tJa82/cp4+PD/z9/Z1ec24MvXr1wv3334+0tDRMmTIFwL9PVz/wwAPo2bNnk72vVrAgExHJCA0Ntd1l7YgkSXYFuqnMmjULqampmDNnDsLDw2E0GqHT6RAXFwer1drk719fkyZNwl/+8hcUFRXBYrFg//79WLZsmdKxbgssyEREjaxr167IzMxEWVmZ3Sr55MmTtv7fO3XqVK195OXlwcvLC+3atQMAbN68GfHx8XjjjTdsY65du4bLly87zHDq1CkMGzbM9vrKlSs4d+4coqOjG/znqiF3U1hcXBzmzp2LDz/8EBUVFXBzc0NsbOwtv2dLwGvIRESNLDo6GtXV1bVWhkuXLoVOp8OIESPs2rOzs+2uAxcWFiI9PR3Dhw+HXq8HAOj1+lqPTL399tu2a8w3WrVqFSorK22vV65ciaqqqlrv3RDe3t5OfxHw8/PDiBEj8MEHHyAtLQ1RUVHw8/O75fdsCbhCJiJqZKNGjcKwYcPwt7/9Dfn5+bj77ruxe/dupKenY86cOejRo4fd+ODgYERGRto99gQASUlJtjH/8z//g/Xr18NoNKJ3797Izs5GZmam7fGrG12/fh0PPvggxo8fjx9++AErVqzAwIEDERMTc8t/vpCQEGRmZmLJkiUICAhA9+7dERYWZuufNGkSxo4dCwBYuHDhLb9fS8GCTETUyFxcXPDJJ5/gxRdfxKZNm5Camopu3bph0aJFmDdvXq3xQ4YMQXh4OJKSkvDzzz+jd+/eWLNmDfr162cb8+abb0Kv1yMtLQ3Xrl3Df/3XfyEzMxORkZEOMyxbtgxpaWl48cUXUVlZiQkTJuCtt95qlC+KWLJkCaZNm2b7WM34+Hi7gjxq1Ci0adMGVqu1UX4BaCl04sZzIERERLegqqoKAQEBGDVqFFavXq10nNsGryETEVGj2r59Oy5evIhJkyYpHeW2whUyERE1igMHDuD777/HwoUL4efn5/ADS8g5rpCJiKhRrFy5Ek899RTat2+PdevWKR3ntsMVMhERkQpwhUxERKQC9X7s6b9dxjVlDqIWZY/1Y6UjEJHK8DlkInLIarXi7NmzaNWqVaM8u0rUUgkhUFZWhoCAANnPPWdBJiKHzp49C5PJpHQMIs0oLCys9c1gv8eCTEQO1XwpQmFhIQwGg8JpiG5fZrMZJpOp1tdx3ogFmYgcqjlNbTAYWJCJGkFdl354lzUREZEKsCATERGpAAsyERGRCrAgExERqQALMhERkQqwIBMREakACzIREZEK8DlkIpIVnJgBF8nrlveTnzKyEdIQaRdXyEQac+TIEURFRcFgMKBVq1YYPnw4jh49qnQsIqoDV8hEGvLtt99i4MCBMJlMSExMhNVqxYoVKzBkyBAcPHgQQUFBSkckIidYkIk0ZP78+fD09ER2djZ8fX0BABMnTkRgYCD++te/YsuWLQonJCJneMqaSEO++uorRERE2IoxAPj7+2PIkCHYsWMHrly5omA6IpLDgkykIRaLBZ6enrXavby8cP36dRw7dkyBVERUHzxlTaQhQUFB2L9/P6qrq6HX6wEA169fx4EDBwAAxcXFTudaLBZYLBbba7PZ3LRhicgOV8hEGvKnP/0JeXl5mDJlCnJzc3Hs2DFMmjQJ586dAwBUVFQ4nZucnAyj0WjbTCZTc8UmIrAgE2nKjBkz8Ne//hUbNmxAnz590LdvX5w+fRrPPvssAMDHx8fp3ISEBJSWltq2wsLC5opNRGBBJtKcV155BRcuXMBXX32F77//HocOHYLVagUABAYGOp0nSRIMBoPdRkTNh9eQiTSoTZs2GDhwoO11ZmYmOnfujF69eimYiojkcIVMpHGbNm3CoUOHMGfOHLi48EeeSK24QibSkL1792LBggUYPnw4fH19sX//fqSmpiIqKgp/+ctflI5HRDJYkIk0pFOnTtDr9Vi0aBHKysrQvXt3vPzyy5g7dy5cXfnjTqRm/Akl0pAePXogIyND6RhE1AAsyEQk61hSJO+4JmoGvMODiIhIBViQiYiIVIAFmYiISAV4DZmIZAUnZsBF8rqlfeSnjGykNETaxRUyERGRCrAgE2nQqVOnEBcXh86dO8PLywu9evXCggULUF5ernQ0InKCp6yJNKawsBChoaEwGo2YOXMm2rZti+zsbCQmJuLIkSNIT09XOiIROcCCTKQx69evx+XLl/H111+jT58+AIBp06bBarVi3bp1uHTpEtq0aaNwSiK6EU9ZE2mM2WwGAHTo0MGu3d/fHy4uLnB3d1ciFhHVgQWZSGOGDh0KAJgyZQqOHj2KwsJCbNq0CStXrsTs2bPh7e3tcJ7FYoHZbLbbiKj5sCATaUxUVBQWLlyIPXv2oH///ujSpQvi4uIwa9YsLF261Om85ORkGI1G22YymZoxNRGxIBNpULdu3TB48GCsWrUKW7ZswRNPPIFXX30Vy5YtczonISEBpaWltq2wsLAZExMRb+oi0piNGzdi2rRpyMvLQ+fOnQEAY8aMgdVqxXPPPYcJEybA19e31jxJkiBJUnPHJaL/wxUykcasWLEC/fv3txXjGjExMSgvL0dOTo5CyYhIDgsykcZcuHAB1dXVtdorKysBAFVVVc0diYjqgQWZSGMCAwORk5ODvLw8u/YPP/wQLi4u6Nevn0LJiEgOryGrkHVQ/wbP/emP8tcAT45b7rTPTaeXnVspaq+6ahy06GTnPpk602mfkJ+KVgVCtr/N2mz5HbQwzzzzDHbt2oVBgwZh5syZ8PX1xY4dO7Br1y5MnToVAQEBSkckIgdYkIk0ZvDgwdi3bx9eeuklrFixAiUlJejevTteeeUVPPvss0rHIyInWJCJNCg0NBQ7d+5slH0dS4qEwWBolH0RkXO8hkxERKQCLMhEREQqwIJMRESkAryGTESyghMz4CJ53dI+8lNGNlIaIu3iCpmIiEgFWvYKWSf/AKx5QliDd13ezvnvOrNnbJWd20f6h9O+ashnvk9y/qwwAFhl+irlH/eFVWb2fXV8BHLOjDed9rnU8XthUVWFbP+imRFO+/Lju8jOrc7Nk+2/HU2ePBlr16512l9UVIROnTo1YyIiqo+WXZCJNGj69OmIiLD/JUUIgRkzZqBbt24sxkQqxYJMpDHh4eEIDw+3a/v6669RXl6ORx99VKFURFQXXkMmagE2bNgAnU6HRx55ROkoROQECzKRxlVWVuKjjz7CgAED0K1bN6XjEJETPGVNpHEZGRkoKSmp83S1xWKBxWKxvTabzU0djYh+hytkIo3bsGED3NzcMH78eNlxycnJMBqNts1kMjVTQiICWJCJNO3KlStIT09HZGQkfH19ZccmJCSgtLTUthUWFjZTSiICWvgpa9fO8o9/ZC16u5mS2JN7LtcK+eeMtSjAVf4h56UBXznt6/3kLNm5Pf+3QZFuG9u3b6/33dWSJEGS6nignIiaDFfIRBqWlpYGHx8fxMTEKB2FiOrAgkykURcvXkRmZiYefvhheHnd2mdRE1HTY0Em0qhNmzahqqqKHwZCdJtgQSbSqLS0NLRv377Wx2gSkTq16Ju6iLQsOzu7UfZzLCkSBoOhUfZFRM5xhUxERKQCml4hu5o6y/aH7vipmZKox8Ac59cTr+73a8Yk//HdU033eFnuePl9x/zv/U323kREN4MrZCIiIhXQ9AqZiG5dcGIGXKRbf2wqP2VkI6Qh0i6ukImIiFSABZlIo7799lvExMSgbdu28PLyQnBwMN566y2lYxGREzxlTaRBu3fvxqhRo9C/f3/Mnz8fPj4+OH36NIqKipSORkROsCATaYzZbMakSZMwcuRIbN68GS4uPBFGdDvgTyqRxmzYsAEXLlzAK6+8AhcXF1y9ehVWq1XpWERUhxa9QrZCJ9v/6q/3OO1bdzhcdq5riZvTvjuebZxPUGqItshrUF9Tiv5yqmz/S2tXy/bfJzn/SsrH84fX8e6X6ui//WRmZsJgMKC4uBgPPfQQ8vLy4O3tjcceewxLly6Fh4eH0hGJyAGukIk05tSpU6iqqsLo0aMRGRmJLVu24IknnsA777yDxx9/3Ok8i8UCs9lstxFR82nRK2QiLbpy5QrKy8sxY8YM213VY8aMwfXr1/Huu+9iwYIFuPPOO2vNS05ORlJSUnPHJaL/wxUykcZ4enoCACZMmGDX/sgjjwBw/qUTCQkJKC0ttW2FhYVNG5SI7HCFTKQxAQEBOH78ODp06GDX3r59ewDApUuOr5tLkgRJkpo8HxE5xhUykcaEhIQAAIqLi+3az549CwBo165ds2ciorqxIBNpzPjx4wEAq1fb353+j3/8A66urhg6dKgCqYioLpo+ZV1VKP+pRIdHdmvwvgOLDjd4rhZVRoTI9le0d/4Y2NKXl8vO7S81/Bnao7vuku03YV+D961W/fv3xxNPPIH3338fVVVVGDJkCLKysvDxxx8jISEBAQEBSkckIgc0XZCJWqp33nkHXbp0QWpqKrZt24auXbti6dKlmDNnjtLRiMgJFmQiDXJzc0NiYiISExOVjkJE9cSCTESyjiVFwmAwKB2DSPN4UxcREZEKsCATERGpAE9ZE5Gs4MQMuEhet7yf/JSRjZCGSLu4QiYiIlKBFr1CrioqrntQC6Lv0F62/9eoHk77libKP0ss9xWJLnX8XljXU8hyX5PZbeuvsnOdp7p9ZWVlYdiwYQ77srOz8cADDzRzIiKqjxZdkIm0bPbs2bj//vvt2nr27KlQGiKqCwsykUYNGjQIY8eOVToGEdUTryETaVhZWRmqqqqUjkFE9cCCTKRRjz/+OAwGAzw8PDBs2DAcPszPXydSM56yJtIYd3d3/PGPf0R0dDT8/PyQm5uLxYsXY9CgQdi3bx/69+/vcJ7FYoHFYrG9NpvNzRWZiMCCTKQ5AwYMwIABA2yvY2JiMHbsWPTr1w8JCQn47LPPHM5LTk5GUlJSc8UkohuwIGuMq6mzbH9uUkenfYPuypOdu63LWw3KdKveutRLtv/QhD5O+6pz5f9MLUXPnj0xevRobN26FdXV1dDr9bXGJCQkYO7cubbXZrMZJpOpOWMStWgsyEQthMlkwvXr13H16lWHXxYhSRIkSVIgGREBvKmLqMX46aef4OHhAR8fH6WjEJEDLMhEGnPx4sVabd999x0++eQTDB8+HC4u/LEnUiOesibSmNjYWHh6emLAgAFo3749cnNzsWrVKnh5eSElJUXpeETkBAsykcY89NBDSEtLw5IlS2A2m9GuXTuMGTMGiYmJ/OhMIhVjQSbSmNmzZ2P27NlKxyCim8SCTESyjiVFOrwrm4gaFwuyAioeCpXt37tildO+SlHXFwYeaUCif3PT1X421f69G34z0JSfHX8dIADkvu/8OWIA8H0vu46981ljIrr98XZLIiIiFWBBJiIiUgGesiYiWcGJGXCRvBplX/kpIxtlP0RaxBUyERGRCrAgE2ncK6+8Ap1Oh+DgYKWjEJEMFmQiDSsqKsKrr74Kb29vpaMQUR14DZlIw55++mk88MADqK6uxq+//qp0HCKSwYKsgMIYq2y/3LPGVsjPvRWVQr4/2+L8OeVnEp+Snev72Y/O+y7W9ZwxNcTevXuxefNm5OTkYNasWUrHIaI6sCATaVB1dTVmzZqFqVOnom/fvvWaY7FYYLFYbK/NZnNTxSMiB1iQiTTonXfeQUFBATIzM+s9Jzk5GUlJSU2Yiojk8KYuIo0pKSnBiy++iPnz56Ndu3b1npeQkIDS0lLbVlhY2IQpiehGXCETacwLL7yAtm3b3vR1Y0mSIElSE6UiorqwIBNpyKlTp7Bq1Sr8/e9/x9mzZ23t165dQ2VlJfLz82EwGNC2bVsFUxKRIzxlTaQhxcXFsFqtmD17Nrp3727bDhw4gLy8PHTv3h0LFixQOiYROcAVsgL0v8kf9nXmTk771v4cLju3KrWDbL/7E+ed9u3ps0V2rhyf4uuy/dUXLzZ431R/wcHB2LZtW632F154AWVlZXjzzTfRo0cPBZIRUV1YkIk0xM/PDw899FCt9r///e8A4LCPiNSBp6yJiIhUgCtkohYgKyurwXOPJUXCYDA0XhgicogrZCIiIhVgQSYiIlIBFmQiIiIV4DVkIpIVnJgBF8mr0faXnzKy0fZFpCUsyAro8fR+2f6Pnu7otM8TZ+rYu3x/3vD7nHf2kd9zmFTptK+ivZvs3FbyuyYiavF4yppIY44fP45x48bhjjvugJeXF/z8/DB48GB8+umnSkcjIhlcIRNpTEFBAcrKyhAfH4+AgACUl5djy5YtiImJwbvvvotp06YpHZGIHGBBJtKY6OhoREdH27XNnDkTISEhWLJkCQsykUrxlDVRC6DX62EymXD58mWloxCRE1whE2nU1atXUVFRgdLSUnzyySfYtWsXYmNjlY5FRE6wIBNp1Lx58/Duu+8CAFxcXDBmzBgsW7bM6XiLxQKLxWJ7bTabmzwjEf0HC7LG6Du0l+0fdFee0z43nV52bqVw3id0slNJAXPmzMHYsWNx9uxZfPTRR6iursb1686/JjM5ORlJSUnNmJCIfo/XkIk0qlevXoiIiMCkSZOwY8cOXLlyBaNGjYIQjn+zSkhIQGlpqW0rLCxs5sRELRsLMlELMXbsWBw6dAh5eY7PkkiSBIPBYLcRUfNhQSZqISoqKgAApaWlCichIkdYkIk05pdffqnVVllZiXXr1sHT0xO9e/dWIBUR1YU3dRFpzPTp02E2mzF48GB06tQJ58+fR1paGk6ePIk33ngDPj4+SkckIgdYkIk0JjY2FqtXr8bKlStRUlKCVq1aISQkBK+99hpiYmKUjkdETrAgE2lMXFwc4uLiGm1/x5IieYMXUTNgQdYYa0A72f73uqxx2lcp5G8piM+PcNrX5p+5snOrZXuJiIg3dREREakACzIREZEK8JQ1EckKTsyAi+TV4Pn5KSMbMQ2RdnGFTEREpAIsyEQac+jQIcycORN9+vSBt7c3unTpgvHjxzv9yEwiUgeesibSmNdeew3ffPMNxo0bh379+uH8+fNYtmwZ7r33Xuzfvx/BwcFKRyQiB1RfkC9NDpft94svcNpnWegvO9f1/x1pUCYl6VsbZft/mOfeZO9dcs3beae5qMnel27O3LlzsWHDBri7/+fvQmxsLPr27YuUlBR88MEHCqYjImdUX5CJ6OYMGDCgVtudd96JPn364MSJEwokIqL64DVkohZACIELFy7Az89P6ShE5AQLMlELkJaWhuLiYsTGxjodY7FYYDab7TYiaj4syEQad/LkSfz5z39GeHg44uPjnY5LTk6G0Wi0bSaTqRlTEhELMpGGnT9/HiNHjoTRaMTmzZuh1+udjk1ISEBpaaltKywsbMakRMSbuog0qrS0FCNGjMDly5fx1VdfISAgQHa8JEmQJKmZ0hHRjViQiTTo2rVrGDVqFPLy8pCZmYnevXsrHYmI6qD6gtxr+nHZ/ve6fO60b1j7WbJzWzUokbJ+fKerbH/uoFVN9t7n/tnFaZ8/+ByyWlRXVyM2NhbZ2dlIT09HeLj8s/xEpA6qL8hEdHPmzZuHTz75BKNGjcJvv/1W64NAJk6cqFAyIpLDgkykMUePHgUAfPrpp/j0009r9bMgE6kTCzKRxmRlZTXq/o4lRcJgMDTqPomoNj72REREpAIsyERERCrAgkxERKQCvIZMRLKCEzPgInnd8n7yU0Y2Qhoi7VJ9QV7Xda9sf6Vwvsj/8o3l8jt/Q7777nfkn2NuqPuij8n2y/2ZK0Vd3+Hc8JMe4S/NlO33f29fg/dNRETyeMqaSGOuXLmCxMREREVFoW3bttDpdFizZo3SsYioDizIRBrz66+/YsGCBThx4gTuvvtupeMQUT2p/pQ1Ed0cf39/nDt3Dh07dsThw4dx//33Kx2JiOqBK2QijZEkCR07dlQ6BhHdJBZkIiIiFeApayICAFgsFlgsFttrs9msYBqilkf1BbnHphmy/bnj326y986Z8WaT7VuO3KNc2Ra97NzpRx6T7e/2cpXTPt/vsuWDkaYlJycjKSlJ6RhELRZPWRMRACAhIQGlpaW2rbCwUOlIRC2K6lfIRNQ8JEmCJElKxyBqsbhCJiIiUgEWZCIiIhXgKWsiDVq2bBkuX76Ms2fPAgA+/fRTFBUVAQBmzZoFo9GoZDwicoAFmUiDFi9ejIKCAtvrrVu3YuvWrQCAiRMnsiATqRALMpEG5efnKx2BiG6S6gty0Hu/yfYPO+D8KxLdnzgvOzeu86EGZQIAk3uJbH+EZ5nTvgMWN9m5pyzOP/ZwddJDsnO7bNwv22+V7SWq7VhSJAwGg9IxiDSPN3URERGpAAsyERGRCqj+lDURKSs4MQMuktct7yc/ZWQjpCHSLq6QiYiIVIAFmUiDLBYLnnvuOQQEBMDT0xNhYWHYs2eP0rGISAYLMpEGTZ48GUuWLMGjjz6KN998E3q9HtHR0fj666+VjkZETuiEEKI+A//bZVxTZ7mt6HsHyvbnj/Fz2tcpq1x2rsvXRxsSiW4je6wfN9m+Dx48iLCwMCxatAhPP/00AODatWsIDg5G+/btsW/fvnrtx2w2w2g0wjTnI15DJroFNT9LpaWlso8QcoVMpDGbN2+GXq/HtGnTbG0eHh6YMmUKsrOz+bWKRCrFgkykMTk5OQgMDKz1m3hoaCgA4OjRowqkIqK68LEnIo05d+4c/P39a7XXtNV84cSNLBYLLBaL7bXZbG6agETkEFfIRBpTUVEBSZJqtXt4eNj6HUlOTobRaLRtJpOpSXMSkT0WZCKN8fT0tFvp1rh27Zqt35GEhASUlpbaNl5rJmpePGVNpDH+/v4oLi6u1X7u3DkAQEBAgMN5kiQ5XFkTUfPgCplIY+655x7k5eXVugZ84MABWz8RqQ9XyA1UnZsn22+qo5+oqYwdOxaLFy/GqlWrbM8hWywWpKamIiwsjNeGiVSKBZlIY8LCwjBu3DgkJCTgl19+Qc+ePbF27Vrk5+dj9erVSscjIidYkIk0aN26dZg/fz7Wr1+PS5cuoV+/ftixYwcGDx6sdDQicoIFmUiDPDw8sGjRIixatEjpKERUTyzIRCTrWFKk7OfvElHj4F3WREREKsCCTEREpAIsyERERCrAgkxERKQCLMhEREQqwIJMRESkAizIREREKsDnkInIISEEANT6kgoiujk1P0M1P1POsCATkUMlJSUAwC+jIGokZWVlMBqNTvtZkInIobZt2wIAfv75Z9l/RNTKbDbDZDKhsLDwtvykMeZXVmPmF0KgrKzM6XeR16h3Qd5j/fiWAhHR7cXF5d+3mBiNxtvyH9QaBoOB+RXE/P9Wn19qeVMXERGRCrAgExERqQALMhE5JEkSEhMTIUmS0lEahPmVxfw3Tyfqug+biIiImhxXyERERCrAgkxERKQCLMhEREQqwIJMRESkAizIRC3E8uXL0a1bN3h4eCAsLAwHDx6UHf/xxx+jV69e8PDwQN++fbFz5067fiEEXnzxRfj7+8PT0xMRERE4deqUKvK/9957GDRoENq0aYM2bdogIiKi1vjJkydDp9PZbVFRUarIv2bNmlrZPDw87Mao+fgPHTq0Vn6dToeRI0faxjTn8d+7dy9GjRqFgIAA6HQ6bN++vc45WVlZuPfeeyFJEnr27Ik1a9bUGnOzP1N1EkSkeRs3bhTu7u7i/fffF8ePHxdPPvmkaN26tbhw4YLD8d98843Q6/Xi9ddfF7m5ueKFF14Qbm5u4l//+pdtTEpKijAajWL79u3iu+++EzExMaJ79+6ioqJC8fyPPPKIWL58ucjJyREnTpwQkydPFkajURQVFdnGxMfHi6ioKHHu3Dnb9ttvvzV69obkT01NFQaDwS7b+fPn7cao+fiXlJTYZT927JjQ6/UiNTXVNqY5j//OnTvF3/72N7F161YBQGzbtk12/E8//SS8vLzE3LlzRW5urnj77beFXq8Xn332mW3MzR6T+mBBJmoBQkNDxZ///Gfb6+rqahEQECCSk5Mdjh8/frwYOXKkXVtYWJiYPn26EEIIq9UqOnbsKBYtWmTrv3z5spAkSXz44YeK579RVVWVaNWqlVi7dq2tLT4+XowePbqxozp0s/lTU1OF0Wh0ur/b7fgvXbpUtGrVSly5csXW1pzH//fqU5CfffZZ0adPH7u22NhYERkZaXt9q8fEEZ6yJtK469ev48iRI4iIiLC1ubi4ICIiAtnZ2Q7nZGdn240HgMjISNv4M2fO4Pz583ZjjEYjwsLCnO6zOfPfqLy8HJWVlbYvzKiRlZWF9u3bIygoCE899ZTtG64aU0PzX7lyBV27doXJZMLo0aNx/PhxW9/tdvxXr16NuLg4eHt727U3x/FviLr+/jfGMXGEBZlI43799VdUV1ejQ4cOdu0dOnTA+fPnHc45f/687Pia/72ZfTZUQ/Lf6LnnnkNAQIDdP6BRUVFYt24dPv/8c7z22mv48ssvMWLECFRXVyuePygoCO+//z7S09PxwQcfwGq1YsCAASgqKgJwex3/gwcP4tixY5g6dapde3Md/4Zw9vffbDajoqKiUf5OOsKvXyQiTUtJScHGjRuRlZVld2NUXFyc7f/37dsX/fr1Q48ePZCVlYUHH3xQiag24eHhCA8Pt70eMGAA7rrrLrz77rtYuHChgslu3urVq9G3b1+Ehobatav5+CuFK2QijfPz84Ner8eFCxfs2i9cuICOHTs6nNOxY0fZ8TX/ezP7bKiG5K+xePFipKSkYPfu3ejXr5/s2DvuuAN+fn748ccfbznz791K/hpubm7o37+/LdvtcvyvXr2KjRs3YsqUKXW+T1Md/4Zw9vffYDDA09OzUf6bOsKCTKRx7u7uCAkJweeff25rs1qt+Pzzz+1WYb8XHh5uNx4A9uzZYxvfvXt3dOzY0W6M2WzGgQMHnO6zOfMDwOuvv46FCxfis88+w3333Vfn+xQVFaGkpAT+/v6NkrtGQ/P/XnV1Nf71r3/Zst0Oxx/496NzFosFEydOrPN9mur4N0Rdf/8b47+pQw2+HYyIbhsbN24UkiSJNWvWiNzcXDFt2jTRunVr26M0jz32mHj++edt47/55hvh6uoqFi9eLE6cOCESExMdPvbUunVrkZ6eLr7//nsxevToJn3s5mbyp6SkCHd3d7F582a7x2rKysqEEEKUlZWJp59+WmRnZ4szZ86IzMxMce+994o777xTXLt2TfH8SUlJIiMjQ5w+fVocOXJExMXFCQ8PD3H8+HG7P6Naj3+NgQMHitjY2FrtzX38y8rKRE5OjsjJyREAxJIlS0ROTo4oKCgQQgjx/PPPi8cee8w2vuaxp2eeeUacOHFCLF++3OFjT3LHpCFYkIlaiLffflt06dJFuLu7i9DQULF//35b35AhQ0R8fLzd+I8++kgEBgYKd3d30adPH/HPf/7Trt9qtYr58+eLDh06CEmSxIMPPih++OEHVeTv2rWrAFBrS0xMFEIIUV5eLoYPHy7atWsn3NzcRNeuXcWTTz55S/+YNmb+OXPm2MZ26NBBREdHi2+//dZuf2o+/kIIcfLkSQFA7N69u9a+mvv4f/HFFw7/PtRkjo+PF0OGDKk155577hHu7u7ijjvusHuGuobcMWkIfv0iERGRCvAaMhERkQqwIBMREakACzIREZEKsCATERGpAAsyERGRCrAgExERqQALMhERkQqwIBMREakACzIREZEKsCATERGpAAsyERGRCrAgExERqcD/Bz6Uz65/6QfeAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 600x700 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataiter = iter(trainloader)\n",
    "images, labels = next(dataiter)\n",
    "images.resize_(128, 784)\n",
    "\n",
    "# Forward pass through the network\n",
    "img_idx = 0\n",
    "logits = model.forward(images)\n",
    "\n",
    "# Predict the class from the network output\n",
    "prediction = F.softmax(logits, dim=1)\n",
    "\n",
    "img = images[0].data\n",
    "view_classification(img.reshape(1, 28, 28), prediction[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset MNIST\n",
       "    Number of datapoints: 60000\n",
       "    Root location: MNIST_data/\n",
       "    Split: Train\n",
       "    StandardTransform\n",
       "Transform: Compose(\n",
       "               ToTensor()\n",
       "               Normalize(mean=0.5, std=0.5)\n",
       "           )"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainloader.dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define a Lighning Module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the LightningModule\n",
    "class MNISTClassifier(pl.LightningModule):\n",
    "    def __init__(self, model):\n",
    "        super().__init__()\n",
    "        self.model = model\n",
    "\n",
    "    def training_step(self, batch, batch_idx):\n",
    "        # training_step defines the train loop.\n",
    "        # it is independent of forward\n",
    "        x, y = batch\n",
    "        x = x.view(x.size(0), -1)\n",
    "        logits = self.model(x)\n",
    "        loss = F.cross_entropy(logits,y)\n",
    "        # Logging to TensorBoard (if installed) by default\n",
    "        self.log(\"train_loss\", loss)\n",
    "        return loss\n",
    "\n",
    "    def test_step(self, batch, batch_idx):\n",
    "        # training_step defines the train loop.\n",
    "        # it is independent of forward\n",
    "        x, y = batch\n",
    "        x = x.view(x.size(0), -1)\n",
    "        logits = self.model(x)\n",
    "        loss = F.cross_entropy(logits, y)\n",
    "        self.log(\"test_loss\", loss)        \n",
    "\n",
    "    def configure_optimizers(self):\n",
    "        optimizer = optim.Adam(self.parameters(), lr=0.001)\n",
    "        return optimizer\n",
    "\n",
    "\n",
    "# init the MNIST Classifier\n",
    "classifier = MNISTClassifier(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "GPU available: False, used: False\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "2023-10-16 19:16:29.089081: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2023-10-16 19:16:29.714684: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "2023-10-16 19:16:29.714762: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "2023-10-16 19:16:29.717316: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
      "2023-10-16 19:16:29.967670: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2023-10-16 19:16:31.699036: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
      "\n",
      "  | Name  | Type       | Params\n",
      "-------------------------------------\n",
      "0 | model | Sequential | 176 K \n",
      "-------------------------------------\n",
      "176 K     Trainable params\n",
      "0         Non-trainable params\n",
      "176 K     Total params\n",
      "0.707     Total estimated model params size (MB)\n",
      "/home/nsanghi/sandbox/apress/drl-2ed/venv/lib/python3.10/site-packages/lightning/pytorch/trainer/connectors/data_connector.py:442: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 20 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fb9096b1bae343da98fe0ae7ff7e5ef7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`Trainer.fit` stopped: `max_epochs=2` reached.\n"
     ]
    }
   ],
   "source": [
    "# train the model (hint: here are some helpful Trainer arguments for rapid idea iteration)\n",
    "trainer = pl.Trainer(max_epochs=2)\n",
    "trainer.fit(model=classifier, train_dataloaders=trainloader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Test the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/nsanghi/sandbox/apress/drl-2ed/venv/lib/python3.10/site-packages/lightning/pytorch/trainer/connectors/checkpoint_connector.py:149: UserWarning: `.test(ckpt_path=None)` was called without a model. The best model of the previous `fit` call will be used. You can pass `.test(ckpt_path='best')` to use the best model or `.test(ckpt_path='last')` to use the last model. If you pass a value, this warning will be silenced.\n",
      "  rank_zero_warn(\n",
      "Restoring states from the checkpoint path at /home/nsanghi/sandbox/apress/drl-2ed/chapter5/lightning_logs/version_7/checkpoints/epoch=1-step=938.ckpt\n",
      "Loaded model weights from the checkpoint at /home/nsanghi/sandbox/apress/drl-2ed/chapter5/lightning_logs/version_7/checkpoints/epoch=1-step=938.ckpt\n",
      "/home/nsanghi/sandbox/apress/drl-2ed/venv/lib/python3.10/site-packages/lightning/pytorch/trainer/connectors/data_connector.py:490: PossibleUserWarning: Your `test_dataloader`'s sampler has shuffling enabled, it is strongly recommended that you turn shuffling off for val/test dataloaders.\n",
      "  rank_zero_warn(\n",
      "/home/nsanghi/sandbox/apress/drl-2ed/venv/lib/python3.10/site-packages/lightning/pytorch/trainer/connectors/data_connector.py:442: PossibleUserWarning: The dataloader, test_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 20 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "33f0c5980b6a46a58bddd2666312ede9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Testing: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">        Test metric        </span>┃<span style=\"font-weight: bold\">       DataLoader 0        </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test_loss         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.1643950194120407     </span>│\n",
       "└───────────────────────────┴───────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test_loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.1643950194120407    \u001b[0m\u001b[35m \u001b[0m│\n",
       "└───────────────────────────┴───────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[{'test_loss': 0.1643950194120407}]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.test(dataloaders=testloader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict with Trained Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAERCAYAAACq8dRTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAmcklEQVR4nO3deVxU9f4/8NcwwLDJqOACOi6pYG5lFIhf1xtXEa9YXhUsA0tTKzW/2kZXJTTD0rTFJe0aLmFabqhXU+kXWYk7VoqGmRCgkpIyIDgs8/n90de5jcwcEIFzPLyej8d51JzP53Pm5RF8z+csczRCCAEiIiKSlYPcAYiIiIgFmYiISBFYkImIiBSABZmIiEgBWJCJiIgUgAWZiIhIAViQiYiIFIAFmYiISAFYkImIiBSABZmISEYajQZTpkypte2tWbMGGo0Gx44dq7LvgAEDMGDAAMvrzMxMaDQarFmzxrLujTfegEajqbV8ZB8LMhGRDbcK263FxcUFfn5+mDJlCvLy8uSOJ6u33noL27dvlzuG6rAgExFJmDt3LtavX4+lS5eid+/eWLFiBYKDg1FcXCx3tLu2b98+7Nu3T7LPrFmzUFJSYrWOBbluOModgIhIyYYMGYKHH34YADBhwgR4eXlh8eLFSEpKwpgxYyr1v3HjBtzd3es7Zo04OztX2cfR0RGOjiwV9YEzZCKiO/C3v/0NAHDhwgWMGzcOHh4eOH/+PMLCwtCoUSM8+eSTAP4szDNnzoTBYIBOp4O/vz8WLVoEew/YS0xMhL+/P1xcXBAQEIADBw5YtWdlZeH555+Hv78/XF1d4eXlhVGjRiEzM9Pm9oqLizFp0iR4eXnB09MTUVFRuHbtmlWf288h23L7OWSNRoMbN25g7dq1lsP548aNw9dffw2NRoNt27ZV2saGDRug0WiQmpoq+V4NHT/2EBHdgfPnzwMAvLy8AADl5eUYPHgw+vTpg0WLFsHNzQ1CCISHh+Prr7/G+PHj8eCDD2Lv3r14+eWXkZubiyVLllht85tvvsGmTZswbdo06HQ6LF++HKGhoThy5Ai6desGADh69CgOHjyIyMhItG7dGpmZmVixYgUGDBiA9PR0uLm5WW1zypQpaNy4Md544w38/PPPWLFiBbKyspCSknJXF2mtX78eEyZMQGBgICZOnAgA6NChA3r16gWDwYDExEQ8/vjjVmMSExPRoUMHBAcH1/h9GwRBRESVJCQkCAAiOTlZXLlyRWRnZ4uNGzcKLy8v4erqKnJyckR0dLQAIF577TWrsdu3bxcAxJtvvmm1fuTIkUKj0YhffvnFsg6AACCOHTtmWZeVlSVcXFzE448/bllXXFxcKWNqaqoAINatW1cpd0BAgCgtLbWsf+eddwQAkZSUZFnXv39/0b9/f8vrCxcuCAAiISHBsi42NlbcXirc3d1FdHR0pTwxMTFCp9OJ69evW9b9/vvvwtHRUcTGxlbqT9Z4yJqISEJISAiaNWsGg8GAyMhIeHh4YNu2bWjVqpWlz3PPPWc1Zvfu3dBqtZg2bZrV+pkzZ0IIgT179litDw4ORkBAgOV1mzZtMHz4cOzduxcVFRUAAFdXV0t7WVkZ8vPz0bFjRzRu3BgnTpyolHvixIlwcnKyyujo6Ijdu3fXYC9UT1RUFEwmEzZv3mxZt2nTJpSXl2Ps2LF19r5qwUPWREQSli1bBj8/Pzg6OqJFixbw9/eHg8N/5zKOjo5o3bq11ZisrCz4+vqiUaNGVuvvv/9+S/tfderUqdL7+vn5obi4GFeuXEHLli1RUlKC+Ph4JCQkIDc31+pcdEFBQaXxt2/Tw8MDPj4+ds8514bOnTvjkUceQWJiIsaPHw/gz8PVvXr1QseOHevsfdWCBZmISEJgYKDlKmtbdDqdVYGuK1OnTkVCQgKmT5+O4OBg6PV6aDQaREZGwmw21/n7V1dUVBRefPFF5OTkwGQy4dChQ1i6dKncse4JLMhERLWsbdu2SE5ORmFhodUs+ezZs5b2vzp37lylbWRkZMDNzQ3NmjUDAGzevBnR0dF49913LX1u3ryJ69ev28xw7tw5DBw40PK6qKgIly5dQlhYWI3/XLdIXRQWGRmJGTNm4LPPPkNJSQmcnJwQERFx1+/ZEPAcMhFRLQsLC0NFRUWlmeGSJUug0WgwZMgQq/WpqalW54Gzs7ORlJSEQYMGQavVAgC0Wm2lW6Y+/PBDyznm261atQplZWWW1ytWrEB5eXml964Jd3d3ux8EvL29MWTIEHz66adITExEaGgovL297/o9GwLOkImIatmwYcMwcOBA/Otf/0JmZiYeeOAB7Nu3D0lJSZg+fTo6dOhg1b9bt24YPHiw1W1PABAXF2fp849//APr16+HXq9Hly5dkJqaiuTkZMvtV7crLS3Fo48+itGjR+Pnn3/G8uXL0adPH4SHh9/1ny8gIADJyclYvHgxfH190b59ewQFBVnao6KiMHLkSADAvHnz7vr9GgoWZCKiWubg4IAdO3Zgzpw52LRpExISEtCuXTssXLgQM2fOrNS/f//+CA4ORlxcHH777Td06dIFa9asQY8ePSx93n//fWi1WiQmJuLmzZv4n//5HyQnJ2Pw4ME2MyxduhSJiYmYM2cOysrKMGbMGHzwwQe18qCIxYsXY+LEiZav1YyOjrYqyMOGDUOTJk1gNptr5QNAQ6ERtx8DISIiugvl5eXw9fXFsGHDsHr1arnj3DN4DpmIiGrV9u3bceXKFURFRckd5Z7CGTIREdWKw4cP48cff8S8efPg7e1t8wtLyD7OkImIqFasWLECzz33HJo3b45169bJHeeewxkyERGRAnCGTEREpADVvu3p7w6j6jIHUYOy3/yF3BGISGF4HzIR2WQ2m3Hx4kU0atSoVu5dJWqohBAoLCyEr6+v5PeesyATkU0XL16EwWCQOwaRamRnZ1d6MthfsSATkU23HoqQnZ0NT09PmdMQ3buMRiMMBkOlx3HejgWZiGy6dZja09OTBZmoFlR16odXWRMRESkACzIREZECsCATEREpAAsyERGRArAgExERKQALMhERkQKwIBMRESkACzIREZECsCATqczx48cRGhoKT09PNGrUCIMGDcLJkyfljkVEVeA3dRGpyIkTJ9CnTx8YDAbExsbCbDZj+fLl6N+/P44cOQJ/f3+5IxKRHSzIRCoye/ZsuLq6IjU1FV5eXgCAsWPHws/PD6+//jq2bNkic0IisoeHrIlU5Ntvv0VISIilGAOAj48P+vfvj127dqGoqEjGdEQkhQWZSEVMJhNcXV0rrXdzc0NpaSlOnTolQyoiqg4esiZSEX9/fxw6dAgVFRXQarUAgNLSUhw+fBgAkJuba3esyWSCyWSyvDYajXUbloiscIZMpCLPP/88MjIyMH78eKSnp+PUqVOIiorCpUuXAAAlJSV2x8bHx0Ov11sWg8FQX7GJCCzIRKoyefJkvP7669iwYQO6du2K7t274/z583jllVcAAB4eHnbHxsTEoKCgwLJkZ2fXV2wiAgsykerMnz8feXl5+Pbbb/Hjjz/i6NGjMJvNAAA/Pz+743Q6HTw9Pa0WIqo/PIdMpEJNmjRBnz59LK+Tk5PRunVrdO7cWcZURCSFM2Qildu0aROOHj2K6dOnw8GBv/JESsUZMpGKHDhwAHPnzsWgQYPg5eWFQ4cOISEhAaGhoXjxxRfljkdEEliQiVSkVatW0Gq1WLhwIQoLC9G+fXu8+eabmDFjBhwd+etOpGT8DSVSkQ4dOmDv3r1yxyCiGuAJJSIiIgVgQSYiIlIAFmQiIiIFYEEmIiJSABZkIiIiBWBBJiIiUgAWZCIVOnfuHCIjI9G6dWu4ubmhc+fOmDt3LoqLi+WORkR28D5kIpXJzs5GYGAg9Ho9pkyZgqZNmyI1NRWxsbE4fvw4kpKS5I5IRDawIBOpzPr163H9+nV899136Nq1KwBg4sSJMJvNWLduHa5du4YmTZrInJKIbsdD1kQqYzQaAQAtWrSwWu/j4wMHBwc4OzvLEYuIqsCCTKQyAwYMAACMHz8eJ0+eRHZ2NjZt2oQVK1Zg2rRpcHd3tznOZDLBaDRaLURUf1iQiVQmNDQU8+bNw/79+9GzZ0+0adMGkZGRmDp1KpYsWWJ3XHx8PPR6vWUxGAz1mJqIWJCJVKhdu3bo168fVq1ahS1btuCZZ57BW2+9haVLl9odExMTg4KCAsuSnZ1dj4mJiBd1EanMxo0bMXHiRGRkZKB169YAgBEjRsBsNuPVV1/FmDFj4OXlVWmcTqeDTqer77hE9H84QyZSmeXLl6Nnz56WYnxLeHg4iouLkZaWJlMyIpLCgkykMnl5eaioqKi0vqysDABQXl5e35GIqBpYkIlUxs/PD2lpacjIyLBa/9lnn8HBwQE9evSQKRkRSeE5ZAUy9+1Z47G//lP6HODZUcvstjlptJJjy0TlWdctR0waybHPJkyx2yakh6JRlpBsb7I2VXoDDczLL7+MPXv2oG/fvpgyZQq8vLywa9cu7NmzBxMmTICvr6/cEYnIBhZkIpXp168fDh48iDfeeAPLly9Hfn4+2rdvj/nz5+OVV16ROx4R2cGCTKRCgYGB2L17t9wxiOgO8BwyERGRArAgExERKQALMhERkQKwIBORpG6xe+WOQNQgsCATEREpQMO+ylojfQOscUxQjTdd3Mz+Z51pk7dKju2q+7fdtgpIZ35YZ/9eYQAwS7SVSd/uC7PE6Ier+ArktMnv221zqOJzYU55iWT7wikhdtsyo9tIjq1Iz5BsvxeNGzcOa9eutduek5ODVq1a1WMiIqqOhl2QiVRo0qRJCAmx/pAihMDkyZPRrl07FmMihWJBJlKZ4OBgBAcHW6377rvvUFxcjCeffFKmVERUFZ5DJmoANmzYAI1GgyeeeELuKERkBwsykcqVlZXh888/R+/evdGuXTu54xCRHTxkTaRye/fuRX5+fpWHq00mE0wmk+W10Wis62hE9BecIROp3IYNG+Dk5ITRo0dL9ouPj4der7csBoOhnhISEcCCTKRqRUVFSEpKwuDBg+Hl5SXZNyYmBgUFBZYlOzu7nlISEdDAD1k7tpa+/SNl4Yf1lMSa1H25ZkjfZ6xGvo7SNzkv8f3WbluXZ6dKju34vzWKdM/Yvn17ta+u1ul00OmquKGciOoMZ8hEKpaYmAgPDw+Eh4fLHYWIqsCCTKRSV65cQXJyMh5//HG4ubnJHYeIqsCCTKRSmzZtQnl5Ob8MhOgewYJMpFKJiYlo3rx5pa/RJCJlatAXdRGpWWpqaq1s51Tc4FrZDhFJ4wyZiIhIAVQ9Q3Y0tJZsD9z1az0lUY4+afbPJ9445F2PSf7rh+fq7vay9NHS2w7/30fq7L2JiO4EZ8hEREQKwIJMRESkACzIRERECsCCTKRSJ06cQHh4OJo2bQo3Nzd069YNH3zwgdyxiMgOVV/URdRQ7du3D8OGDUPPnj0xe/ZseHh44Pz588jJyZE7GhHZwYJMpDJGoxFRUVEYOnQoNm/eDAcHHggjuhfwN5VIZTZs2IC8vDzMnz8fDg4OuHHjBsxms9yxiKgKDXqGbIZGsv2tqw/abVt3LFhyrGO+k922+16pnW9QqommyKhRW10K+2aCZPsba1dLtj+ss/9IyqczB1Xx7teqaL/3JCcnw9PTE7m5uXjssceQkZEBd3d3PPXUU1iyZAlcXFzkjkhENnCGTKQy586dQ3l5OYYPH47Bgwdjy5YteOaZZ/DRRx/h6aeftjvOZDLBaDRaLURUfxr0DJlIjYqKilBcXIzJkydbrqoeMWIESktLsXLlSsydOxedOnWqNC4+Ph5xcXH1HZeI/g9nyEQq4+rqCgAYM2aM1fonnngCgP2HTsTExKCgoMCyZGdn121QIrLCGTKRyvj6+uL06dNo0aKF1frmzZsDAK5ds33eXKfTQafT1Xk+IrKNM2QilQkICAAA5ObmWq2/ePEiAKBZs2b1nomIqsaCTKQyo0ePBgCsXm19dfq///1vODo6YsCAATKkIqKqqPqQdXm29LcSHRvarsbb9ss5VuOxalQWEiDZXtLc/m1gS95cJjm2p67m99Ce3HO/ZLsBB2u8baXq2bMnnnnmGXzyyScoLy9H//79kZKSgi+++AIxMTHw9fWVOyIR2aDqgkzUUH300Udo06YNEhISsG3bNrRt2xZLlizB9OnT5Y5GRHawIBOpkJOTE2JjYxEbGyt3FCKqJp5DJiIiUgAWZCIiIgVgQSYiIlIAFmQiIiIFYEEmIiJSgAZ9lXV5Tm7VnRoQbYvmku1XQzvYbVsSK30vsdQjEh2q+FxY1V3IUo/JbLf1quRY+6nuXSkpKRg4cKDNttTUVPTq1aueExFRdTTogkykZtOmTcMjjzxita5jx44ypSGiqrAgE6lU3759MXLkSLljEFE18RwykYoVFhaivLxc7hhEVA0syEQq9fTTT8PT0xMuLi4YOHAgjh3j968TKRkPWROpjLOzM/75z38iLCwM3t7eSE9Px6JFi9C3b18cPHgQPXv2tDnOZDLBZDJZXhuNxvqKTERgQSZSnd69e6N3796W1+Hh4Rg5ciR69OiBmJgYfPnllzbHxcfHIy4urr5iEtFtNEIIUZ2Of3cYVddZqBY4GlpLtqfHtbTb1vf+DMmxH7f5qkaZqlLVbU/vXfOTbP96tP1HP1akS/+Z5LLf/EW9v+eYMWOwdetWFBcXQ6vVVmq3NUM2GAwoKCiAp6dnfUYlUhWj0Qi9Xl/l7xJnyEQNhMFgQGlpKW7cuGHzHwWdTgedTidDMiICeFEXUYPx66+/wsXFBR4eHnJHISIbWJCJVObKlSuV1v3www/YsWMHBg0aBAcH/toTKREPWROpTEREBFxdXdG7d280b94c6enpWLVqFdzc3LBgwQK54xGRHSzIRCrz2GOPITExEYsXL4bRaESzZs0wYsQIxMbG8qsziRSMBZlIZaZNm4Zp06bJHYOI7hBPJhERESkAZ8gyKHksULL9wPJVdtvKRFUPDDxeg0R/ctJUvjfV+r1r/vlt/G+2HwcIAOmfdJUc6/VxahVbV+a9xkREd4IzZCIiIgVgQSYiIlIAFmQiIiIFYEEmIiJSABZkIpWbP38+NBoNunXrJncUIpLAgkykYjk5OXjrrbfg7u4udxQiqgJveyJSsZdeegm9evVCRUUFrl69KnccIpLAgiyD7HCzZLvUvcZmSI+9G2VVPBk71WT/PuWXY5+THOv15S/2265UdZ8x1cSBAwewefNmpKWlYerUqXLHIaIqsCATqVBFRQWmTp2KCRMmoHv37tUaYzKZYDKZLK+NRmNdxSMiG1iQiVToo48+QlZWFpKTk6s9Jj4+HnFxcXWYioik8KIuIpXJz8/HnDlzMHv2bDRr1qza42JiYlBQUGBZsrOz6zAlEd2OM2QilZk1axaaNm16x+eNdToddDpdHaUioqqwIBOpyLlz57Bq1Sq89957uHjxomX9zZs3UVZWhszMTHh6eqJp06YypiQiW3jImkhFcnNzYTabMW3aNLRv396yHD58GBkZGWjfvj3mzp0rd0wisoEzZBlo/5De7euMrey2rf0tWHJseUILyXbnZy7bbdvfdYvkWCkeuaWS7RVXrtR421R93bp1w7Zt2yqtnzVrFgoLC/H++++jQ4cOMiQjoqqwIBOpiLe3Nx577LFK69977z0AsNlGRMrAQ9ZEREQKwBkyUQOQkpIidwQiqgJnyERERArAgkxERKQALMhEREQKwHPIRCSpW+xeOOjc5I5BVC8yFwyV7b1ZkGXQ4aVDku2fv9TSbpsrLlSxden2jEEP22/sKr3lIF2Z3baS5k6SYxtJb5qIqMHjIWsilTl9+jRGjRqF++67D25ubvD29ka/fv2wc+dOuaMRkQTOkIlUJisrC4WFhYiOjoavry+Ki4uxZcsWhIeHY+XKlZg4caLcEYnIBhZkIpUJCwtDWFiY1bopU6YgICAAixcvZkEmUigesiZqALRaLQwGA65fvy53FCKygzNkIpW6ceMGSkpKUFBQgB07dmDPnj2IiIiQOxYR2cGCTKRSM2fOxMqVKwEADg4OGDFiBJYuXWq3v8lkgslksrw2Go11npGI/osFWWW0LZpLtve9P8Num5NGKzm2TNhvExrJoSSD6dOnY+TIkbh48SI+//xzVFRUoLTU/mMy4+PjERcXV48JieiveA6ZSKU6d+6MkJAQREVFYdeuXSgqKsKwYcMghO1PVjExMSgoKLAs2dnZ9ZyYqGFjQSZqIEaOHImjR48iI8P2URKdTgdPT0+rhYjqDwsyUQNRUlICACgoKJA5CRHZwoJMpDK///57pXVlZWVYt24dXF1d0aVLFxlSEVFVeFEXkcpMmjQJRqMR/fr1Q6tWrXD58mUkJibi7NmzePfdd+Hh4SF3RCKygQWZSGUiIiKwevVqrFixAvn5+WjUqBECAgLw9ttvIzw8XO54RGQHCzKRykRGRiIyMrLWtncqbjAv8CKqByzIKmP2bSbZ/nGbNXbbyoT0JQXRmSF225r8J11ybIVkKxER8aIuIiIiBWBBJiIiUgAWZCIiIgVgQSYiIlIAFmQilTl69CimTJmCrl27wt3dHW3atMHo0aPtfmUmESkDr7ImUpm3334b33//PUaNGoUePXrg8uXLWLp0KR566CEcOnQI3bp1kzsiEdmg+IJ8bVywZLt3dJbdNtM8H8mxjv/veI0yyUnbWC/Z/vNM5zp77/yb7vYbjTl19r50Z2bMmIENGzbA2fm/PwsRERHo3r07FixYgE8//VTGdERkj+ILMhHdmd69e1da16lTJ3Tt2hVnzpyRIRERVQfPIRM1AEII5OXlwdvbW+4oRGQHCzJRA5CYmIjc3FxERETY7WMymWA0Gq0WIqo/LMhEKnf27Fm88MILCA4ORnR0tN1+8fHx0Ov1lsVgMNRjSiJiQSZSscuXL2Po0KHQ6/XYvHkztFqt3b4xMTEoKCiwLNnZ2fWYlIh4UReRShUUFGDIkCG4fv06vv32W/j6+kr21+l00Ol09ZSOiG7HgkykQjdv3sSwYcOQkZGB5ORkdOnSRe5IRFQFxRfkzpNOS7Z/3OYru20Dm0+VHNuoRonk9ctHbSXb0/uuqrP3vvSfNnbbfMD7kJWioqICERERSE1NRVJSEoKDpe/lJyJlUHxBJqI7M3PmTOzYsQPDhg3DH3/8UemLQMaOHStTMiKSwoJMpDInT54EAOzcuRM7d+6s1M6CTKRMLMhEKpOSkiJ3BCKqAd72REREpAAsyERERArAgkxERKQALMhEREQKoPiLuta1PSDZXibsf6b45t1l0ht/V7r5gY+k72OuqYfDTkm2S/2Zy0RVz3Cu+Wes4DemSLb7fHywxtsmIiJpnCETqUxRURFiY2MRGhqKpk2bQqPRYM2aNXLHIqIqsCATqczVq1cxd+5cnDlzBg888IDccYiomhR/yJqI7oyPjw8uXbqEli1b4tixY3jkkUfkjkRE1cAZMpHK6HQ6tGzZUu4YRHSHWJCJiIgUgIesiQgAYDKZYDKZLK+NRqOMaYgaHsUX5A6bJku2p4/+sM7eO23y+3W2bSlSt3KlmrSSYycdf0qyvd2b5XbbvH5IlQ5GqhYfH4+4uDi5YxA1WDxkTUQAgJiYGBQUFFiW7OxsuSMRNSiKnyETUf3Q6XTQ6XRyxyBqsDhDJiIiUgAWZCIiIgXgIWsiFVq6dCmuX7+OixcvAgB27tyJnJwcAMDUqVOh1+vljEdENrAgE6nQokWLkJWVZXm9detWbN26FQAwduxYFmQiBWJBJlKhzMxMuSMQ0R1SfEH2//gPyfaBh+0/ItH5mcuSYyNbH61RJgAwOOdLtoe4FtptO2xykhx7zmT/aw9Xxz0mObbNxkOS7WbJViIikgsv6iIiIlIAFmQiIiIFYEEmIiJSABZkIiIiBWBBJlIhk8mEV199Fb6+vnB1dUVQUBD2798vdywiksCCTKRC48aNw+LFi/Hkk0/i/fffh1arRVhYGL777ju5oxGRHRohhKhOx787jKrrLPcUbRc/yfbMEd5221qlFEuOdfjuZE0i0T1kv/mLOtv2kSNHEBQUhIULF+Kll14CANy8eRPdunVD8+bNcfDgwWptx2g0Qq/Xo6CgAJ6ennWWl0jtqvu7xBkykcps3rwZWq0WEydOtKxzcXHB+PHjkZqayscqEikUCzKRyqSlpcHPz6/SJ/HAwEAAwMmTJ2VIRURVUfw3dRHRnbl06RJ8fHwqrb+17tYDJ25nMplgMpksr41GY90EJCKbOEMmUpmSkhLodLpK611cXCzttsTHx0Ov11sWg8FQpzmJyBoLMpHKuLq6Ws10b7l586al3ZaYmBgUFBRYFp5rJqpfPGRNpDI+Pj7Izc2ttP7SpUsAAF9fX5vjdDqdzZk1EdUPzpCJVObBBx9ERkZGpXPAhw8ftrQTkfJwhlxDFekZku2GKtqJ6srIkSOxaNEirFq1ynIfsslkQkJCAoKCgnhumEihWJCJVCYoKAijRo1CTEwMfv/9d3Ts2BFr165FZmYmVq9eLXc8IrKDBZlIhdatW4fZs2dj/fr1uHbtGnr06IFdu3ahX79+ckcjIjv41ZlEMqjLr86sLfzqTKLawa/OJCIiuoewIBMRESkACzIREZECsCATEREpAAsyERGRArAgExERKQALMhERkQKwIBMRESkAv6mLiGy69Z1Btz+kgojuzK3foaq+h4sFmYhsys/PBwA+jIKolhQWFkKv19ttZ0EmIpuaNm0KAPjtt98k/xFRKqPRCIPBgOzs7Hvyqz+ZX161mV8IgcLCQrvPIr+l2gX5XvjuXSKqPQ4Of15iotfr78l/UG/x9PRkfhkx/5+q86GWF3UREREpAAsyERGRArAgE5FNOp0OsbGx0Ol0ckepEeaXF/PfuWo/D5mIiIjqDmfIRERECsCCTEREpAAsyERERArAgkxERKQALMhEDcSyZcvQrl07uLi4ICgoCEeOHJHs/8UXX6Bz585wcXFB9+7dsXv3bqt2IQTmzJkDHx8fuLq6IiQkBOfOnVNE/o8//hh9+/ZFkyZN0KRJE4SEhFTqP27cOGg0GqslNDRUEfnXrFlTKZuLi4tVHyXv/wEDBlTKr9FoMHToUEuf+tz/Bw4cwLBhw+Dr6wuNRoPt27dXOSYlJQUPPfQQdDodOnbsiDVr1lTqc6e/U1USRKR6GzduFM7OzuKTTz4Rp0+fFs8++6xo3LixyMvLs9n/+++/F1qtVrzzzjsiPT1dzJo1Szg5OYmffvrJ0mfBggVCr9eL7du3ix9++EGEh4eL9u3bi5KSEtnzP/HEE2LZsmUiLS1NnDlzRowbN07o9XqRk5Nj6RMdHS1CQ0PFpUuXLMsff/xR69lrkj8hIUF4enpaZbt8+bJVHyXv//z8fKvsp06dElqtViQkJFj61Of+3717t/jXv/4ltm7dKgCIbdu2Sfb/9ddfhZubm5gxY4ZIT08XH374odBqteLLL7+09LnTfVIdLMhEDUBgYKB44YUXLK8rKiqEr6+viI+Pt9l/9OjRYujQoVbrgoKCxKRJk4QQQpjNZtGyZUuxcOFCS/v169eFTqcTn332mez5b1deXi4aNWok1q5da1kXHR0thg8fXttRbbrT/AkJCUKv19vd3r22/5csWSIaNWokioqKLOvqc///VXUK8iuvvCK6du1qtS4iIkIMHjzY8vpu94ktPGRNpHKlpaU4fvw4QkJCLOscHBwQEhKC1NRUm2NSU1Ot+gPA4MGDLf0vXLiAy5cvW/XR6/UICgqyu836zH+74uJilJWVWR6YcUtKSgqaN28Of39/PPfcc5YnXNWmmuYvKipC27ZtYTAYMHz4cJw+fdrSdq/t/9WrVyMyMhLu7u5W6+tj/9dEVT//tbFPbGFBJlK5q1evoqKiAi1atLBa36JFC1y+fNnmmMuXL0v2v/XfO9lmTdUk/+1effVV+Pr6Wv0DGhoainXr1uGrr77C22+/jW+++QZDhgxBRUWF7Pn9/f3xySefICkpCZ9++inMZjN69+6NnJwcAPfW/j9y5AhOnTqFCRMmWK2vr/1fE/Z+/o1GI0pKSmrlZ9IWPn6RiFRtwYIF2LhxI1JSUqwujIqMjLT8f/fu3dGjRw906NABKSkpePTRR+WIahEcHIzg4GDL6969e+P+++/HypUrMW/ePBmT3bnVq1eje/fuCAwMtFqv5P0vF86QiVTO29sbWq0WeXl5Vuvz8vLQsmVLm2Natmwp2f/Wf+9kmzVVk/y3LFq0CAsWLMC+ffvQo0cPyb733XcfvL298csvv9x15r+6m/y3ODk5oWfPnpZs98r+v3HjBjZu3Ijx48dX+T51tf9rwt7Pv6enJ1xdXWvl79QWFmQilXN2dkZAQAC++uoryzqz2YyvvvrKahb2V8HBwVb9AWD//v2W/u3bt0fLli2t+hiNRhw+fNjuNuszPwC88847mDdvHr788ks8/PDDVb5PTk4O8vPz4ePjUyu5b6lp/r+qqKjATz/9ZMl2L+x/4M9b50wmE8aOHVvl+9TV/q+Jqn7+a+Pv1KYaXw5GRPeMjRs3Cp1OJ9asWSPS09PFxIkTRePGjS230jz11FPitddes/T//vvvhaOjo1i0aJE4c+aMiI2NtXnbU+PGjUVSUpL48ccfxfDhw+v0tps7yb9gwQLh7OwsNm/ebHVbTWFhoRBCiMLCQvHSSy+J1NRUceHCBZGcnCweeugh0alTJ3Hz5k3Z88fFxYm9e/eK8+fPi+PHj4vIyEjh4uIiTp8+bfVnVOr+v6VPnz4iIiKi0vr63v+FhYUiLS1NpKWlCQBi8eLFIi0tTWRlZQkhhHjttdfEU089Zel/67anl19+WZw5c0YsW7bM5m1PUvukJliQiRqIDz/8ULRp00Y4OzuLwMBAcejQIUtb//79RXR0tFX/zz//XPj5+QlnZ2fRtWtX8Z///Meq3Ww2i9mzZ4sWLVoInU4nHn30UfHzzz8rIn/btm0FgEpLbGysEEKI4uJiMWjQINGsWTPh5OQk2rZtK5599tm7+se0NvNPnz7d0rdFixYiLCxMnDhxwmp7St7/Qghx9uxZAUDs27ev0rbqe/9//fXXNn8ebmWOjo4W/fv3rzTmwQcfFM7OzuK+++6zuof6Fql9UhN8/CIREZEC8BwyERGRArAgExERKQALMhERkQKwIBMRESkACzIREZECsCATEREpAAsyERGRArAgExERKQALMhERkQKwIBMRESkACzIREZECsCATEREpwP8HqPzIexduCrYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 600x700 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "logits = model.forward(img[None,])\n",
    "model.eval()\n",
    "\n",
    "# Predict the class from the network output\n",
    "prediction = F.softmax(logits, dim=1)\n",
    "\n",
    "view_classification(img.reshape(1, 28, 28), prediction[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that model is able to correctly predict the digit after training while before training it predicting all digits with almost equal probability i.e. it was randomly predicting the digit. We also see that PyTorch Lighning reduced the bolierplate code a lot. We just need to define the forward pass and loss calculation for train and test step. And just need to define the initial configuration of the optimizer.\n",
    "\n",
    "There is no need to define individual training loops and manage the streaming of data through the dataloader. There is no need to code the backward step through the optimizer.\n",
    "\n",
    "The code you have written in MNSITClassifier is specific to your problem with almost zero boilerplate code."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
